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(54) Title: METHOD AND APPARATUS FOR ENHANCING AN IMAGE USING DATA OPTIMIZATION AND SEGMENTATION 
(57) Abstract 



A method and system are disclosed for generating enhanced images of 
multiple dimensional data using a depth-buffer segmentation process (218). 
The method and system operate in a computer system modify the image by 
generating a reduced dimensionality image data set from a multi-dimensional 
image by formulating a set of projection paths through image points selected 
from the multi-dimensional image (212). selecting an image point along each 
projection path, analyzing each image point to detcnnine spacial similarities 
with at least one other point adjacent to the selected image point in a given 
dimension, and grouping the image point with the adjacent point (222) or 
spacial similarities between the points is found thereby defining the data set. 
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METHOD AND APPARATUS FOR ENHANCING 
AN IMAGE USING DATA OPTIMIZATION 
AND SEGMENTATION 

BACKGROUND OF THE INVENTION 
The present invention relates generally to data image processing and, more 
specifically, to generating enhanced digital images utilizing image segmentation 
processes that involve depth buffer information in association with point selection 
projection images. 

The development of high resolution 3D (three-dimensional) imaging techniques, 
such as magnetic resonance imaging (MRI), computed tomography (CT), and rotation X- 
ray CT angiography (XRCTA), have led to situations where high volumes of image data 
are generated in the study of individual patients. In many cases, such as in 3D magnetic 
resonance angiography (MRA) and 3D CT angiography (CT A), the amount of detail per 
two-dimensional slice is small, but the high resolution detail of the full image volume is 
required in the diagnostic process. In one example, arterial beds are generally sparse, so 
there is typically a large amoimt of nonvascular image data present in a 3D angiographic 
image. As the resolution and volume coverage of MRA examinations have increased 
recently, having adequate means to effectively and eflBciently review the available image 
information is a significant challenge for radiologists. Currently, there are three primary 
display techniques used for clinical evaluation of MRA data. The first technique is to 
display selected original cross-sectional MRA images. The second technique is to 
display oblique planar reformatted images. The last technique is to display a maximum 
intensity projection (MIP) image. The following discussion focuses mostly on MRA 
images; however, the discussion is equally applicable to other 3D angiography techniques 
such as CTA and XRCTA and to other volumetric visualization techniques in which 
selectable groups of image points are of interest. 

The original cross-sectional images, also referred to as slices, contain the 
maximum amount of information on a local level. Each image displays the transverse 
segments and cross sections of vessels that intersect the plane of the image. Sometimes, 
the vascular detail of interest may be observed in a single image. More often, because 
of the intricate paths of the vessels, many images must be viewed, and the information 
firom each image must be integrated to fomiulate an understanding of the structure of 
interest. This method is inefficient since it requires a single slice to be displayed, which 
slice contains very little global information about any single intricate vessel in the overall 
vascular network. 



wo 00/63S31 PCT/USOO/10532 



The oblique planar reformatted images provide improved efficiency over the 
original images by displaying image planes that follow a segment of a vessel of interest 
through the three-dimensional image data. Each individual image still provides only 
local information and a complete view of the vascular bed is not obtained. 
5 Volume rendering consists of the projection or rendering of an entire 3D image 

volume onto a single two-dimensional image. The volume is projected along parallel or 
diverging lines through the three-dimensional volume onto a two-dimensional image. 
The intensities along the projection line are transformed according to some specified 
transformation. Such rendering, in a variety of forms, has become very useful in assisting 

10 the observer in the rapid and efficient interpretation of the large amounts of image data 
originally obtained. A simple form of volume rendering, which is also intuitive, is an X- 
ray-like summation of the image densities or intensities. Initial attempts at MRA found 
that when the summation type of volume rendering was used, the background signal was 
too large to the point of masking a majority of vascular details during the summation 

15 process. The display capabilities were improved to the point where useful vascular 
details were observable when the rendering was performed by selecting only the 
maximum image value encountered along each projection line. This approach is known 
as the maximum intensity projection (MBP) algorithm and has been used in the 
development of a variety of MRA techniques. Other forms of volume rendering, such 

20 as assigning opacities and translucensies to certain image values or regions of image 

values, have been applied and found useful. However, the MIP algorithm is dominant 
because of its simplicity and consistency in generating quality images. 

The MIP algorithm is successfiil to the extent that the signal fi-om vessels is 
greater than the signal of the surrounding tissues. In regions where vessels do not appear 

25 overlapped, the MIP algorithm is sufficient to display vessels that are hyperintense 
relative to the variations in the overall background. Unfortunately, the MIP algorithm 
does not provide any information about vessels that are hidden below the intensity of 
other structtires. Because of the sparseness of vessels, there is a significant amount of 
vessel detail that is not hidden and the MIP performs very well in increasing the amount 

3 0 of vessel detail observed in a single image. 

Although there is a loss of information in a MIP image, the MIP algorithm 
provides a large amount of useful information in a single display. The information 
density, or information content per image element, is much higher in a MIP image than 
in the original 3D source data. In other words, although the source image data contains 

3 5 more total infomiation, including some small arteries that would not appear in the MEP 
image, the density of vessel mfomiation in the source image data (i.e. the percentage of 
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image elements associated with vessels) is lower than in the MIP image. As discussed 
later, many investigators have tried to develop algorithms to overcome the limitations of 
the MIP algorithm, and, although these have been viewed as improvements, the 
improvements have not been sufficient for any of these algorithms to replace the MIP 
algorithm. The advantages of the MIP algorithm typically outweigh the disadvantages 
found therein. These advantages include reduced dynamic range, generally consistent 
image display, and improved signal difference to noise ratio (SDNR or contrast to noise 
ratio) for vessels that appear in the MIP. The artifacts in the MIP, although very real, are 
also well understood and can be 'Yead through.*' 

The MIP display contains a large amount of global information about the vascular 
system. The appearance of a MIP image is quite similar to that of an X-ray angiogram, 
but there are several differences. The MIP simply selects the image element with the 
maximum intensity along each projection line, while the X-ray projection is a summation 
of all densitometric information along each projection line. The MIP image is therefore 
a flat display having no depth information and no information about the thickness of the 
vessel through which the projection line passed, while in X-ray angiography the vessel 
brightness or darkness depends directly on the length of the X-ray proj ection path through 
the vessel. Thus, in regions where several vessels are overlapping in the MIP image, it 
is difficult to resolve specific vessel segments. Further, because the signal intensity is not 
related to the projection path length through the vessel, an increase in vessel signal is not 
observed for foreshortened vessels. There are other limitations that exist in the MDP 
algorithm. Statistical nosie properties associated with the signal of the tissue background 
cause the signal level of the background in the MIP to increase with the projection 
volume thickness. Consequently, small vessels with signal levels that are slightly above 
the local backgroimd signal level may be lower than other background image elements 
along the projection line and, therefore, may not be observed in the MIP images. The 
limitations of the MIP can make it necessary to also review the original low information 
density cross-sectional images. When used in combination, the two display formats are 
a means for detecting and evaluating many types of vascular pathology. The review 
process, however, is rather time consimiing, especially for complex cases. 

Various attempts have been made to overcome the deficimcies that exist in the 
MIP algorithm. The appearance of small vessels in the MIP image can be improved by 
the use of zero filled interpolation (ZFI). The problem addressed by ZFI is that the 
arbitrary positions of the vessels relative to the reconstruction grid cause small vessels 
or vessel borders to be poorly rendered in the image due to the weighting of the voxel 
sensitivity function. ZFI reconstmcts the image on a grid that is finer than the acquisition 



wo 00/(i3831 



PCT/USOO/10532 



4 

grid, thereby reducing vessel jaggedness in the MIP image caused by partial volume 
effects and improving vessel continuity and visibility in the MIP image for subvoxel- 
sized vessels. 

Other attempts to overcome the problems inherent in the MIP algorithm and to 
improve the two-dimensional rendering of MRA image data range from simple 
modifications of the MIP algorithm, to attempts at complete segmentation and redisplay 
of the vascular information contained in the original MRA data. These attempts include 
utilizing the traveling or sliding slab MIP technique; preprocessing the original MRA 
image data before application of the MIP algorithm; line-based segmentation, where the 
vessel voxels are segmented from the background image data, which is also similar to 
segmentation work perfonned in other fields; intensity based segmentation, where the 
vessel voxels are segmented based upon relative intensities and proximity in 3D; and 
interactive intensity based segmentation, where vessel voxels that cannot be segmented 
based upon intensity are displayed over a wider gray scale range than those that can be 
segmented. 

MRA display c^abilities are also useful in a number of intracranial applications. 
There are various disease processes for which the diagnosis can benefit from improved 
image display capabilities. These diagnostic capabilities include predicting the need for 
thrombolysis in stroke conditions, diagnosing vasculitis and other occlusive diseases, 
identifying intracranial tumors and arterial venous malformations, and performing 
preoperative assessments. Further, MRA and other 3D angiographic unages provide 
useful assistance for surgical procedures. The display of catheter angiograms and/or 
MRA image data have been found to be important aids during aneurysm surgery. The 
usefulness of the display of 3D angiographic image data during surgery can be enhanced 
by the display of the angiographic images in conjunction with 3D images of other types 
of anatomy such as adjacent bone structures or critical organs. 

SUMMARV OF THE INVRNTTON 
According to the present invention, a method and system are disclosed for 
enhancing and segmenting a multi-dimensional image based upon the depth buffer (or 
"Z-buffer") that is generated in conjunction with a maximum intensity projection (or 
related point selection projection) operation through the multi-dimensional image. This 
enhancement and segmentation is referred to as the depth-buffer segmentation (DBS) 
process. The DBS process segments an MRA image volume into regions having a 
likelihood of vessel occupation and regions that are unlikely to contain vessels. The 
process reduces the total image volume to a much smaller volume so that the amount of 
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territory that a user must cover is greatly reduced. In addition, projection images of the 
vascular information found within the data become much more refined and visible when 
the background regions have been removed. 

The DBS process merges the global properties of the MIP process with the local 
properties of continuity to achieve a high specificity in the segmentation of vessel voxels 
from backgroimd voxels while maintaining a high sensitivity. The segmentation provides 
an accurate and robust view of the vessel structure and filters out most of the non-vessel 
regions in the image. After the vascular detail is segmented, it can then be converted 
back to image data for display - for example, as a densitometric svuiunation of the MRA 
image data resembling that of an X-ray angiogram. In an optional step, the dynamic 
range of the display is reduced by hollowing out the vessel structures and displaying the 
vascular details as X-ray projection through hollow tubes. Further, the image may be 
modified by adding a degree of shading so that the 3D vessel structure is apparent in a 
manner not visible in X-ray images. Such image shading was not possible in the MIP 
process. 

In one embodiment, the method generates a reduced dimensionality image data 
set Grom a multi-dimensional image by formulating a set of projection paths through 
image points selected firom the multi-dimensional image, selecting an image point along 
each projection path, analyzing each image point to determine spacial similarities with 
at least one other point adjacent to the selected image point in a given dimension, and 
grouping the image point with the adjacent point or spacial similarities between the 
points is found thereby defining the data set. The method, within the analyzing step, the 
step of determining similarity of brightness between the image point and the adjacent 
point. Further, the analyzing step also determines similarity of smoothness between the 
image point and the adjacent point. In one example, the smoothness is determined by 
using a least squares fit of adjacent image points. In yet an alternative embodiment, the 
method fiirther includes selecting another image point along the projection path and 
performing the analyzing and grouping steps on the newly selected image point. Further 
still, the method may also convert the grouped and ungrouped image points into a multi- 
dimensional image and then perform region growing within the converted multi- 
dimensional image or perform hollowing-out of the multi-dimensional image for image 
enhancement. Another image enhancement steps include removing all pixels that are 
surrounded on each side by an adjacent pixel prior to displaying the image of the merged 
and immerged image points. 

The system further defines an image processing apparatus that comprises means 
for defining a set of projection paths through a multi-dimensional image, means for 
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selecting at least one point, along each projection path, based upon a specified criterion, 
means for formulating an array of projection values corresponding to the positions of 
selected points along their respective projection path, and means for grouping a selected 
number of projection values based upon their proximity to other projection values. The 
apparatus essentially is a programmable computer system that loads a particular software 
program capable of implementing the steps of the method claims within a computer 
architecture environment for manipulating the data and processing it for imaging, whether 
the imaging is in a printed image or a displayed image, such as on a computer monitor. 
Further still, the method may be implemented in a computer program product for sale or 
distribution, whether that product is in a portable medium or in a fixed mediimi or remote 
medium that is communicated via a communications source, such as the Internet or other 
communication means known to those skilled in the art. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The foregoing will become more fiilly apparent from the following description 
and appended claims, taken in conjunction with the accompanying drawings. 
Understanding that these drawings depict only typical embodiments of the invention and 
are, therefore, not to be considered limiting of its scope, the invention will be described 
and explained with additional specificity and detail through the use of the accompanying 
drawings in which: 

Figure 1 illustrates a block diagram of the imaging system utilizing depth-buffer 
(or Z-buffer) segmentation in accordance with the present invention; 

Figure 2 illustrates a flow diagram of the method steps implemented by the 
imaging apparatus of Figure 1 in rendering DBS images in accordance with the present 
invention; 

Figure 3 illustrates a standard axial MIP image of a patient with a large 
ophthalmic artery aneurysm; 

Figure 4 illustrates a depth-buffer or Z-buffer array corresponding to Figure 3 in 
which brightness is proportional to the "Z" location of each point selected for the MIP 
along each projection line; 

Figure 5 depicts a plot of the intensity values of the data firom the projection lines 
for the 30 points indicated by the short line that crosses the middle cerebral artery as 
shown in Figure 4. The z«locations of the maximum intensity values projected by the 
MIP algorithm are indicated by the vertical marks. 

Figures 6 through 8 illustrate an expanded view of the small region of the Z- 

buffer array of Figure 4 where Figure 6 illustrates example Une segments used for the 
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least squares fit roughness calculation. Figure 7 illustrates image brightness inversely 
proportional to the roughness values in accordance with the present invention, and Figure 
8 illustrates the same region with brightness set proportional to group size after grouping 
low roughness pixels that have similar depth values - i.e. have connectivity in the Z- 
buffer; 

Figures 9 through 1 1 illustrate a series of images where each image represents 
where Figure 9 is a brightness inversely proportional to the roughness value in 
accordance with the present invention. Figure 10 is a brightness based upon number of 
points connected in the Z-buffer, where connectivity is based upon local smoothness and 
proximity in the depth or "Z" direction, and Figure 1 1 is a brightness proportional to 
nimiber of points connected after connectivity has been enhanced by the 3D region 
growing process; 

Figures 12 through 14 illustrate part of an original 3D image with the 
corresponding segmented image and also the segmented image overlayed on the original 
image. 

Figure 1 5 illustrates the performance of the segmentation based on a vessel and 
non-vessel classification of the segmented image of Figures 1 2 through 14 as performed 
by an independent observer; 

Figures 16 through 18 illustrate stereo pair X-ray-like densitometric projections 
(DPs) of the DBS segmentation corresponding to the image of Figure 3, where Figure 16 
is the DP of all segmented vessels, Figure 17 is the anterior circulation, and Figure 18 is 
a combination of DP and shaded surface display of a hollow representation of the DBS 
segmentation of the anterior and posterior circulations; 

Figures 19 through 22 illustrate stereo pair X-ray-like DPs, based on the 
principles of the present invention, of the intracranial circulation of a patient with a 
posterior communicating artery aneurysm and a basilar tip aneurysm ; 

Figures 23 through 24 illustrate stereo pair X-ray-like DPs of a contrast enhanced 
abdominal aorta in accordance with the present invention; and. 

Figures 25 through 26 illustrate a set of CTA images including (A) axial collapse 
of segmented data structures and (B) shaded hollow-body reprojection in accordance with 
the present invention. 

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 

It will be readily understood that the components of the present invention, as 
generally described and illiistrated in the figures herein, could be arranged and designed 
in a wide variety of different configurations. Thus, the following more detailed 
description of the embodiments of the system and method of the present invention, as 

SUBSTTTUTE SHEET (RULE 26) 
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represented in Figures 1 through 26, is not intended to limit the scope of the invention, 
as claimed, but is merely representative of the presently preferred embodiments of the 
invention. Likewise, the following description focuses on the capability of the present 
invention to isolate blood vessels for enhanced angiographic imaging, but this is not 
5 intended to limit the potential uses for which the invention may be of value to only those 
examples given. 

The presently described embodiments of the invention will be best understood by 
reference to the drawings, wherein like parts are designated by like nimierals throughout. 
Figure 1 illustrates a block diagram of an imaging apparatus 10 in accordance 

1 0 with the prjesent invention. Imaging apparatus 1 0 comprises an imaging device 12, a Z- 
buffer MIP processor 14, an image processor 16, a storage system 18, ou^ut device 20, 
and image viewing device 22. Each element is connected with at least one other element 
within apparatus 10 or, alternatively, each element is coimected to a common bus 23 that 
allows communication to and from each element with any other element within apparatus 

15 10. . 

Imaging device 12 typically is a magnetic resonance imaging (MRI) system or a 
computed tomography (CT) system or a rotational X-ray CT angiography (XRCTA) 
system used in generating 3D images in a non-invasive manner of a subject, such as an 
object or a patient, which images can be stored in computer manipulable form for 

20 subsequent processing and display. Z-Bufifer MIP processor 14 and storage system 18 
are typically selected as a personal computer-type system or work station. One such 
system includes a personal computer based on the Advanced Micro Devices (AMD) 
Athlon 650 megahertz CPU with a storage system comprising 128 megabytes of RAM 
and a 20 gigabyte long-tenri storage hard drive. Alternatively, the computer system can 

25 be selected from other known computer systems as previously mentioned, such as for 
example, a work station like that provided by Sim Computers, such as their UltraSpark 
Platform. The computer system further incorporates the image processor 1 6. One image 
processor suitable for use is provided by Mitsubishi and is known as the VP 5003D 
graphic accelerator. The personal computer system operates in a Windows NT operating 

3 0 system environment. Other operating systems include Windows 98 and Windows 2000, 
which are both provided by Microsoft of Redmond, Washington. Further, the invention 
can be implemented on an Apple compatible system running under OS 9 and configured 
in a comparable manner to the personal computer mentioned above. 

Output device 20 can include such items as a printer system or removable storage 

35 device. The printing systems typically utilized as output devices offer a high quality 
output such as those found in digital photography printers. Other printers, including low 
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resolution laser printers, would also be acceptable in some situations. Typically, 
however, the use of better resolution in printing is desirable. Image viewing device 22 
typically is a video monitor. The monitor typically is a high resolution monitor given the 
large amount of image detail that is of interest The depth-buffer segmentation process 
makes it feasible to include a graphical user interface to allow a user to select different 
viewing options, different angles of view, rotational viewing options, and other 
animation-type viewing options, which enhance and optimize the viewing needs of the 
user. 

The operation of apparatus 10, as well as the process for converting raw image 
data iiLformation into a useful depth-buffer segmentation (DBS) image, is shown in the 
flow diagram of Figure 2 and is presented in greater detail below in accordance with the 
present invention. In order to begin processing an image, an image must be generated or 
taken of the subject matter to be examined. The image or images can come from various 
imaging modalities. These modalities can be selected fixim magnetic resonance imaging 
systems or other 3D imaging systems. Other imaging modalities include computed 
tomography (CT), computed tomography angiography (CTA), andX-ray CT angiography 
(XRCTA) as well as other types of 3D imaging systems known to those skilled in the art. 
Further, it is the image data itself that is useful and processed as will be shown below. 

In experiments made to evaluate the process of the present invention, image data 
acquisition weis performed as part of an aneurysm imaging protocol. The imaging was 
performed on a 1.5 Tesla (T) General Electric model Signa MRI scanner with actively 
shielded gradients. The gradient system operates with a maximum strength of 23 milli- 
Tesla/meter (mT/m) and a slew rate of 77 milli-Tesla/meter/millisecond (mT/m/ms) on 
all three axes for the MRA sequence. An optimized cylindrical birdcage transmit/receive 
coil with an RF mirror endcap is used. Generally, the image is acquired in any desired 
technique, one of which is illustrated in the acquisition of step 212, which uses a three- 
dimensional time-of-flight (TOF) sequence. The 3D TOF sequence incorporates imaging 
techniques such as abbreviated magnetization transfer, flow compensation in the 
frequency and slice-encode directions, and RF spoiling, which are understood by those 
skilled in the art. For this image acquisition process, the imaging parameters used and 
that are understood by those skilled in the art were: repetition time (Tr) = 48 msec, echo 
time (Te) = 5.3 msec, readout bandwidth (RBW) = ± 16 kHz, "in-plane" matrix = 512 x 
192 image elements, field of view (FOV) = 220 x 165 mm^. 

The next step, step 214, although optional for the present invention, applies zero 
filled interpolation in all 3 image directions. After the optional interpolation, step 216 
applies MIP processing to the 3D image data. MEP processing involves generating a 2D 

SUBSTITUTE SHEET (RULE 26) 
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image from the 3D image, where the image value selected for each point in the 2D unage 
is the maximum (or minimum) image value found along a corresponding line through the 
3D image. The MIP projection lines may all be parallel, as in a parallel ray MIP image, 
or the lines may be non-parallel and for example might diverge from a single source point 
as in a divergent ray MIP image or in curvilinear space as welL 

In step 2 1 8 an array is generated from the origmal 3D image data by determining 
the depth or Z-positions of the points included in the MIP image. In this Z-buffer array, 
the value of each element is related to the distance of the corresponding point in the MBP 
image from the 2D image plane (i.e. the location of the maximum (or minimum) intensity 
value along each projection line). Figure 3 illustrates a standard axial MIP image of a 
patient having an ophthalmic artery aneurysm. The image was acquh^ed in a 512 x 256 
X 64 image format and then interpolated to 1024 x 1024 x 128, which is shown in step 
214. The unage is generally considered to have high quality, but the artifacts inherent in 
the MIP algorithm, such as vessels obscured by other vessels, are visible. Figure 4 
displays the depth-buffer (or Z-buffer) array corresponding to the axial MIP image of 
Figure 3. As noted, the Z-buffer array values, displayed here as levels of brightness, are 
proportional to the depth or "Z" positions of the pomts that appear in the MIP unage of 
Figure 3. 

The generation of the MIP with the Z-buffer may actually be formed in any 
direction through the original 3D MRA image data. A convenient direction that is chosen 
in the samples previously mentioned and that is used in the current embodiment is to 
perform the MIP in the original slice selection direction. Typically, the path will be along 
a defined line taken for evaluation, whether it be a straight line, a divergent line, or a 
curvilinear line, as determined by the operator at the time the evaluation is performed. 

Figure 5 illustrates a plot of signal intensities along the projection lines in the 
original 3D image data for 30 pomts taken from Figure 3. The vertical marks on the 30 
plots of Figure 5 show the positions and values of the points that projected into the MIP 
image m Figure 3. Each of these lines corresponds to a point in the vertical bar that 
crosses the right middle cerebral artery shown in the display of the Z-buffer array of 
Figure 4. 

Two observations are made looking at these illustrations. The first is that the 
vessels exhibit very high continuity in position in the "Z" location. This continuity is 
used in a primitive vessel detection algorithm that generates a value based on the 
smoothness in at least one direction around a point. For example, the intensities in the 
uoaage of Figure 7 are uiversely related to the mimmum value obtamed from the first 
order fit of 5 pomts in the four principal directions around each point of Figure 6, which 
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is a magnified view of the region indicated with a small rectangle in Figure 4. The four 
principal directions are defined as 0, 4S» 90 and 133 degrees from horizontal in the 2D 
image plane. For those points where the ^ value is low in at least one direction (smooth 
neighborhood), the im^e is bright. As will be shown, the vessel segments visible in the 
NOP of Figure 3 are all characterized by smoothness and continuity. In view of this, the 
process of performing depth-buffer segmentation consists in grouping image points based 
on this smoothness and continuity. The major vessels are characterized by large groups 
of points. 

The second observation is that the apparent width of the vessel "continuity" 
region in the Z-buffer array display is noticeably wider than the apparent width of the 
vessels in the MIP image (Figure 4 as compared to Figure 3). This difference in width 
is explained by noting that the slow flow of blood near the vessel edges results in very 
low signals. Yet, the signals at the vessel edges can still be sufficiently strong to cause 
many of these edge points to project in the MIP image. Hence, the edge points are 
present in Figure 3 even though their brightness is scarcely distinguishable from the 
background. In the Z-buffer array of Figure 4, the edge points are seen to have the same 
depth as the brighter central points of Figure 3, confirming that they pertain to the same 
anatomical structure. 

The apparatus and process of the present invention measures the local roughness 
for each element within the Z-buffer array in accordance with step 220. The process 
measures the roughness (or smoothness), as described below, in the Z-buffer array. In 
most areas of the Z-buffer array, only pixel groups from vessels exhibit smoothness in 
one or more directions. To measure the smoothness, the process performs a low order 
polynomial least squares fit to Z-buffer array values (MIP image Z-locations) in the four 
principal directions around each point in the Z-buffer array. The four directions are 
shown in the upper left hand quadrant 600 of Figure 6. Beginning with the brightest 
point in the MIP image, every point in the MIP image is tested for classification as vessel 
based upon local smoothness in the corresponding Z-buffer array. If the element is fotmd 
to be locally "smooth" the process tests all eight neighbors in two dimensions for 
additional possible vessel elements. If the neighboring elements are also "smooth** and 
if they are close in "Z," they are added to the group and their neighbors are added to the 
list for testing. The process continues until all neighbors to the group are too "rough" or 
too separated in "Z" to be connected. 

The process then considers the next nonclassified, brightest point in the MIP 
image and tests whether it is a "smooth" element. If this element is smooth, it is defined 
as the next group and the method repeats the process of looking for additional members 
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to add to this new group. The process continues until all elements in the Z-bufFer array 
have been tested. 

Figure 6 illustrates an expanded view of a small region of the Z-bufTer array of 
Figure 4. The four small line segments, segments 610-616, show the length (S pixels) 
of the segment used for the first order least squares fit. Points 610 and 614 are outside 
vessels and yield a high value for the fit in any direction. Point 612 is within a vessel, 
but yields a high value except for the direction parallel to the vessel. Point 616 is within 
the vessel and yields a low value for three out of four possible directions. Figure 7 
illustrates the same region as Figure 6, where the image brightness of each pixel is 
inversely proportional to its minimum calculated roughness value, or value, as 
specified below. The value for the fit in the jth direction (j=l ,2,3,4) for the ith element 
in the Z-buffer array is: 




where Zj^^ is the Z-buffer value (depth-buffer value) for element number k (k=l ,2,3,4,5) 
along the line having element i at its center and direction j. The Pyj^ corresponds to a 
predicted value for the same element determined from a low order polynomial fit to the 
values along the line. The brightness in the array shown in Figure 7 is then inversely 
related to the minimimi value. For each point i and all corresponding lines j=l ,2,3,4: 

and brightness, Vj, is 

V, =A/(B+zf) 

where the constants A and B are empirically determined. The brightness values displayed 
in Figure 7 tend to increase with the likelihood that the element is part of a vessel. 

Other methods of measuring smoothness in the MIP Z-bufFer could be used. The 
current method involves performing a low order least squares fit to a small nxmiber of 
image elements centered at a particular point. In the current embodiment, the process 
vrtilizes a first order fit along five points, and an example of applying the fit in four 
principal directions is shown in Figure 6. Altematively, a different nimiber of points, a 
different order fit, and a different number of directions may be used in measuring local 
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roughness or smoothness as desired by the system designer. Or the process could be 
omitted completely in some implementations. 

As may be deduced from Figure 6, only points that are contained Avitfain vessels 
will have a very low value. Points just outside the vessel edges generally experience 
erratic Z-buffer depth values at some point along each of the lines selected for the fit, 
resulting in high ^ values. Because a vessel only requires smoothness in one direction, 
the process saves only the minimum value of the for each point. The minimum 3^ 
image is illustrated in. Figure 9 illustrates an image where the brightness is inversely 
proportional to the minimum value of the least squares fits of short line segments along 
the principal directions. 

It is then possible to look throughout the minimum image for regions of 
connected "smoothness" as performed in Step 222 of Figure 2 in order to determine 
regions that are most likely to be vessels. Such regions are indicated by the grouped 
image elements of Figures 12 through 14. Figures 12 through 14 illustrates points in the 
Z-buffer tiiat are grouped together based upon their low roughness values and proximity. 
The brightness shown in Figures 12 through 14 is proportional to the number of points 
in each groiq>. 

In the implementation of step 222, the process performs a grouping operation 
where each data point is considered in a specific order. The use of the MIP image implies 
that the bright parts of the original three dimensional image data were the most important. 
As such, the process performs the connectivity operations by selecting the brightest image 
element in the MIP image. This element is tested for a low value in the corresponding 
minimum array. If the minimiun is below a predefined threshold, the point is 
selected as a possible vessel and the neighboring points in the 2D MIP image are tested. 
The brightest element and all of its neighbors and their neighbors, and so forth, which 
satisfy the coimectivity criterion, are added to the group. The process then considers the 
next non-classified brightest point and determines which remaining neighboring points 
satisfy the connectivity criterion to form another group. The process continues until all 
points have been tested. To be connected in this embodiment of the invention, the 
minimimi ^ value must be below a given threshold and the Z-position must be within a 
small distance of the currently considered point. For example, in this illiistration, the 
threshold value for equaled 1 .0 and a step of +/- 2 in Z was allowed. Other values may 
be selected by the designer according to the needs of the designer, but this step size 
recognizes that larger vessels occasionally project from different regions within the 
vessel, and the larger value improves the character of the connectedness. Figures 16 
through 18 illustrates an image of the points that have been grouped based upon the 
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proximity in "Z" and minimum roughness. The intensity of the display is proportional 
to the number of points in each group. It is shown that some vessels, although obviously 
"connected" in real life, are not connected by this implementation of the process (e.g. the 
middle cerebral artery is not connected to the trifurcadon area). These disconnects 
happen because only one point in a thick vessel is projected by the MIP algorithm, and 
the thickness of the vessel allows larger jumps in "Z" than are accepted by the 
connectivity criteria used in this example. 

Once all the contiguous groups of minimum roughness are determined in the MIP 
Z-buffer array, there are still many vessels that are crossed by other vessels and remain 
broken. Further, only one point per projection line is included in the vessel group. It is 
therefore useful to map each group of vessel points back to their original three 
dimensional image positions where a region growing algorithm is utilized for each point 
in each group to add additional points from the three dimensional image data to each 
vessel group as shown in step 224. In step 224, to complete the process of connecting 
all vessels, the groups are mapped back to the three dimensional image space, and all 
points from the two dimensional groups are considered again in the same order. All 
neighboring points in the 3D data are tested based upon their intensity being a selected 
(e.g. two standard deviations) above the avemge background value. Points that are two 
standard deviations or more above the average background would have projected in the 
MIP image had they not been obscured by other brighter structures, such as vessels. By 
adding these points to the groups, the process fills out the vessels in 3D and also connects 
most branches that were "disconnected" by being obscured by other vessels. An image 
of points connected by the process in step 224 is illustrated in 11 . 

As indicated by step 226, extraneous groups are removed from the set of grouped 
image points created in step 224. For example, noise groups can be removed 
automatically by eliminating all groups with very small numbers of elements. In this 
example, all groups with less than 45 elements were considered to be noise and were 
removed. All the major vessels subtend more than 45 elements, and very few noise 
groups have more than 45 elements (voxels). Where groups with large numbers of 
voxels are not of interest, such as regions of fat or bone or organ tissue in angiogmphic 
imaging, they can be eliminated by a manual step, that allows the user to select the groups 
to be removed and cause those groups not to display. 

A qualitative evaluation of the results of the segmentation process can be 
performed by comparing the elements segmented as belonging to vessels with those seen 
in the original cross-sectional images. The DBS process performs well in classifying 
vessel and non-vessel voxels. The misclassifications consist of the smallest vessels that 
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did not appear in the MEP and the centers of the carotid arteries that were dark in the 
original 3D data. Figures 12 through 14 show an example of points in the original 3D 
image that, as shown in Figure 1 S, were manually classified as vessel and non-vessel by 
an independent observer. The points segmented by the DBS process of the present 
invention as vessels and non-vessels are shown as white and black regions of points in 
Figures 12 through 14. The manually classified points appear as vAdte points in these 
same figures. Figure 1 5 illustrates graphs of the performance of the segmentation based 
on the vessel and non-vessel classification of the segmented image of Figures 1 2 through 
14 as performed by an independent observer. The graphs represents the vessel inclusion 
sensitivity as measured by the nimiber of voxels in a group of a few hundred and of 
xlOOO. Line910representsvery small vessels means only see in local projections. Line 
912 represents small vessels as seen in the NOP images. Line 914 represents the medium 
size secondary bifiircations, M2, A2. Line 916 represents large vessels, such as internal 
carotid and middle cerebral arteries. 

Once all of the data elements have been classified by the DBS algorithm as groups 
of structures such as vessels, any of the previously mentioned variety of display 
techniques may be utilized, including MIPs and shaded surface renderings. By displaying 
the original MRA intensities of only those points classified as vessel, it is straightforward 
to perform densitometric summations along lines to the original data. The resulting 
images, as shown in Figures 16 through 18, look much like subtraction X-ray 
angiograms, but the dynamic range is so large that it is hard to portray the vessels at a 
single window/level setting on a video display. More specifically. Figure 10 illustrates 
a stereo pair X-ray-like densitometric projection through DBS process images firom the 
aneurysm study of Figure 3. Figure 16 is a stereo pair of the fiill intracranial circulation 
while Figure 17 illustrates a stereo pair of vessels connected to the anterior and middle 
cerebral arteries. It is difficult to show the fiill dynamic range of intensities contained in 
the images of Figures 16 and 17. The dynamic range is reduced by removing all points 
internal to the DBS segmented structures. X-ray-like densitometric projection using the 
hollow DBS process in accordance with the present invention is shown in Figure 1 8. In 
Figure 18 some surface shading is also added to the X-ray-like projection image. In 
Figure 18, the intracranial carotid arteiy element 1010 is visible below the aneurysm. 
Thus, the process manipxilates the dynamic range of information for display as shown in 
step 228 of Figure 2 to enhance the resolution of the vessel structures of interest during 
the display. 

As just noted, one example of performing dynamic range reduction is to eliminate 
all data points within the image that have neighbors in every direction. This results in an 
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image where the vessels appear as hollow tubes. Typical displays of densitometric 
projections through the hollow vessel im^es are shown in Figure 1 8 and in Figures 1 9 
through 26. The characterization of the image may be further modified by adding an 
amount of shading to each vessel surface thereby enabling the observer to determine 
whether the vessels are near or far, and the vessel orientation is more discemable as well. 
Once the final manipulation of the dynamic range is performed, the process, as shown in 
step 230, displays the processed image on an imaging device, such as a video monitor 
and/or prints the image on a printing apparatus. Optional display processes are 
contemplated that may yield more visual information. The present embodiment of the 
invention may not eliminate all noise, but the noise is reduced to the point that the useful 
data is easily recognized over these noise points. 

Figures 1 9 through 22 illustrate stereo pair X-ray-like densitometric reprojections 
of the results of the DBS segmentation of points fix)m the 3D image of a patient with a 
communicating artery (PCOM) aneurysm and a basilar tip aneurysm. Figure 19 is the 
cranial view of the PCOM aneurysm while Figure 20 is the caudal view. Figure 21 
highlights the posterior commvinicating artery aneurysm 1108 while Figure 22 illustrates 
the highlight of a small basilar tip aneurysm, which is behind tip 1 1 1 0 and is shown from 
two different orientation. 

Figures 23 through 24 illustrates a stereoscopic X-ray-like densitometric 
reproj ection through a renal artery study in accordance with the present invention. Figure 
23 A shows the anterior view while Figure 24 illustrates the posterior view. 

Lastly, Figures 25 through 26 represents an example of CTA images acquired at 
relatively low resolution on a helical CT scaimer. Segmentation is performed with the 
DBS process in accordance with the present invention. Figure 25 depicts the axial 
collapse of the segmented data structures, while Figure 26 illustrates shaded hollow-body 
reprojections of the aortic aneurysm. 

The DBS process as described and presented in the present invention results in 
an image segmentation process that is readily applicable to magnetic resonance 
angiography (MRA), computed tomography angiography (CTA), rotational X-ray 
angiography (XRCTA), and other medical and non-medical applications. Since the DBS 
process is based upon the generic MIP algorithm, the application of the DBS process can 
be extended wherever the MIP algorithm is currently being used. The utility of the DBS 
process can be enhanced to include display options that allow the user to toggle between 
the DBS process and the preliminary MIP process, as well as other forms of volume 
rendering. The DBS process is also applicable to such fields as computer assisted 
screening and image interpretation based upon segmented anatomy. 
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The present invention may be embodied in other specific forms without departing 
from its spirit or essential characteristics. The described embodiments are to be 
considered in all respects only as illustrative and not restrictive. The scope of the 
invention is, therefore, indicated by the appended claims, rather than by the foregoing 
description. All changes which come within the meaning and range of equivalency of the 
claims are to be embraced within their scope. 

What is claimed is: 
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1 . A method of generating a reduced-dimensionality image data set from a 
multi-dimensional image, comprising: 

A formulating a set of projection paths through image points selected from 

the multi-dimensional image; 
B selecting an image point along each projection path; 
C analyzing each image point to determine spatial similarities with at least 

one other point adjacent to the selected image point in any given 

dimension; and 

D grouping the image point with the adjacent point where spatial 
similarities between the points is foimd thereby defining the data 
set. 

2. The method according to claim 1 wherein step C comprises determining 
similarity of brightness between the image point and the adjacent point. 

3. The method according to claim 1 wherein step C comprises determining 
similarity of smoothness between the image point and the adjacent point. 

4. The method according to claim 3 wherein smoothness is determined using 
a least squares fit of adjacent image points. 

5. The method according to claim 1 wherein steps C and D are repeated for 
each point adjacent the selected image point. 

6. The method according to claim 1 further comprising £ selecting another 
image point along the projection path and performing steps C and D on the newly 
selected image point. 

7. The method according to claim 1 wherein step B formulates a Z-buffer 

array. 

8. The method according to claim 1 further comprising F converting the 
grouped and ungrouped image points into a multi-dimensional image. 

9. The method according to claim 8 further comprising G performing region 
growing within the converted multi-dimensional image. 

1 0. The method according to claim 8 further comprising H hollowing out the 
multi-dimensional image. 

1 1 . The method according to claim 10 wherein step H comprises removing 
all pixels that are surrounded on each side by an adjacent pixel. 

12. The method according to claim 1 further comprising step I displaying an 
image of the merged and unmerged image points. 
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1 3 . The method according to claim 1 wherein the image data set is a magnetic 
resonance derived image set. 

1 4. The method according to claim 1 wherein the image data set is a computer 
tomography derived image set. 

15. An image processing method comprising: 

A formulating a projection image of a multiple dimensional image based on 
a set of image elements of the multi-dimensional image; 

B projecting a path for each selected image element wherein an image 
element along each projection line is selected based upon a 
specified criterion; 

C formulating a depth-buffer array corresponding to the projection image, 
wherein the value of each array element is the depth or position of 
the selected image element along the projection path in the multi- 
dimensional image; and, 

D analyzing the depth-buffer array to detect adjacent points that are close in 
position along their depth-buffer projection paths. 

1 6. The method according to claim 1 5 wherein the depth buffer comprises a 
reduced dimensionality over the multi-dimensional image. 

1 7. The method according to claim 1 5 wherein the selection criterion of step 
B is based on whether the image element is a maximum or minimum value contained 
along the projection line. 

1 8 . The method according to claim 1 5 wherein the depth-buffer array is based 
upon z-positions of the projection image. 

1 9. The method according to claim 1 5 wherein step D comprises determining 
a roughness level in several directions around each element in the depth-buffer array. 

20. The method according to 19 wherein roughness is determined by 
evaluating a mathematical fit of depth-buffer array values along each line in the several 
directions. 

21. The method according to 19 further comprising saving a minimum 
roughness value of the several roughness values obtained at each point. 

22. The method according to 1 9 wherein the roughness value is a Chi square 
value for each point. 

23. A method of separating a multi-dimensional image into selected groups 
of image elements for image processing, comprising: 
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A formulating a depth buffer of points, using selected image elements, 
based on depth or position along an associated projection path for 
each selected image element in the multiple-dimensional image; 

B for each point in the depth buffer, grouping selected adjacent points of 
image elements based on proximity of position along the depth 
buffer; 

C mapping the points of a selected group back to the respective image 
element of the multiple-dimensional image; and 

D connecting additional selected image points to the group, wherein steps 
C and D are repeated for each selected group of image points 
detected in the depth buffer. 

24. The method according to claim 23 wherein, for a selected group of 
mapped and connected image points, each image element is selected for display based 
upon defined criteria. 

25. The method according to claim 24 wherein a criterion to formulate the 
projection image is based on: 

selecting a maximum or minimum value of the image elements contained along 
each projection line. 

26. The method according to claim 23 further comprising step E comprising 
compensating for variations in sensitivity along projection paths to enhance the proj ection 
image. 

27. The method according to claim 23 wherein step A further comprises 
selecting the depth along a z-axis of each image element projected to form the depth 
buffer. 

28. The method according to claim 23, after formulating the depth buffer array 
in step A, further comprising applying a process to the depth buffer to enhance the depth- 
buffer array based upon expected properties of adjacent points in the depth buffer. 

29. The method according to claim 28 wherein the process comprises 
measuring array element roughness in a plurality of directions around each array element 
in the depth-buffer array. 

30. The method described in 29 wherein the roughness measures are minimum 
Chi Square values and wherein the measuring step comprises: 

evaluating each depth-buffer array element to determine its minimum 
Chi Square value based on a plurality of Chi Squares for each 
element; and 
saving the minimum Chi Square value. 
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31. An image processing method comprising: 

A defining a set of projection paths through a multi-dimensional image; 
B for each projection path, selecting at least one point based upon a 
specified criterion; 

C formulating an array of projection values corresponding to the position of 
selected points along their respective projection path; and, 

D grouping a selected number of projection values based upon their 
proximity to other projection values. 

32. The method according to claim 31 wherein the specified criterion is a 
selected threshold value. 

33. The method according to claim 32 wherein the threshold value is 
determined from an average background value of all points in a subset of points selected 
from the multi-dimensional image. 

34. The method according to claim 31 wherein the specified criterion is a 
maximum value along the projection path. 

35. The method according to claim 31 wherein the specified criterion is a 
minimum value along the projection path. 

36. The method according to claim 3 1 further comprising: 
selecting at least one group of projection values; 

selecting the points of the multi-dimensional image corresponding to the 

projection values of the selected group; and 
generating an image based upon the selected multi -dimensional image points. 

37. The method according to claim 31 wherein the projection paths are 
curvilinear. 

38. The method according to claim 31 wherein the projection paths are 
divergent from a point of origin. 

39. The method according to claim 3 1 wherein the proximity is based on at 
most two image element positions from the selected point. 

40. The method according to claim 3 1 wherein the proximity is based on point 
adjacency. 

4 1 . The method according to claim 3 1 fiirther comprising selecting a second 
set of projection paths and repeating steps B-D. 

42. The method according to claim 41 fiirther comprising combining the 
points of the multi-dimensional image selected from the projection values from at least 
one group selected from the first set of project paths with the projection values from at 
least one group selected from the second set of projection paths. 
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43. The method according to claim 41 further comprising: 
selecting at least one group of projection values; 

selecting the points of the multi-dimensional image corresponding to the 

projection values of the selected group; and 
generating an image based upon the selected multi-dimensional image points. 

44. The method according to claim 3 1 further comprising manipulating image 
groups for enhanced display. 

45. The method according to claim 44 wherein the image manipulation 
consists of hollowing image structures. 

46. The method according to claim 45 wherein the hollowing step comprises 
removing all voxels in a group that are surrounded in all directions by adjacent voxels in 
the same group. 

47. The method according to claim 3 1 further comprising step G displaying 
the resulting multidimensional image. 

48. The method according to claim 47 wherein the display comprises 
summation of the resulting multi-dimensional image along projection lines. 

49. The method according to claim 47 wherein the image display comprises 
shading volume surfaces. 

50. The method according to claim 3 1 wherein the multidimensional image 
set is a magnetic resonance derived image. 

5 1 . The method according to claim 3 1 wherein the multidimensional image 
set is a computed tomography derived image. 

52. The method according to claim 3 1 wherein the multidimensional image 
set is a rotational X-ray computed tomography image. 

53. The method according to claim 31 comprising the step of identifying 
bifurcations and branches of groups segmented from the multidimensional image. 

54. The method according to claim 3 1 comprising labeHng selected segmented 
components. 

55. The method according to claim 31 further comprising the step of 
identifying pathological processes based upon the shape of the segmented group of image 
elements. 

56. An image processing apparatus comprising: 

means for defining a set of projection paths through a multi-dimensional image; 
means for selecting at least one point, along each projection path, based upon a 



specified criterion; 
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means for formulating an array of projection values corresponding to the 
positions of selected points along their respective projection path; and, 

means for grouping a selected nimiber of projection values based upon their 
proximity to other projection values. 

57. The apparatus according to claim 56 wherein the specified criterion is a 
selected threshold value. 

58. The apparatus according to claim 57 wherein the threshold value is 
determined firom an average background value of all points in a subset of points selected 
from the multi-dimensional image. 

59. The apparatus according to claim 56 wherein the specified criterion is a 
maximum value along the projection path. 

60. The apparatus according to claim 56 wherein the specified criterion is a 
minimum value along the projection path. 

61 - The apparatus according to claim 56 fiirther comprising: 

means for selecting at least one group of projection values; 

means for selecting the points of the multi-dimensional image corresponding to 

the projection values of the selected group; and 
means for generating an image based upon the selected multi-dimensional image 

points. 

62. The apparatus according to claim 56 wherein the array comprises a depth 

buffer. 

63. The apparatus according to claim 56 wherein the array comprises a 
curvilinear buffer. 

64. The apparatus according to claim 56 wherein the projection paths are 
divergent from a point of origin- 

65 . The apparatus according to claim 56 wherein the proximity is based on at 
most two image element positions from the selected point. 

66. The apparatus according to claim 56 wherein the proximity is based on 
point adjacency. 

67. The apparatus according to claim 56 fiirther comprising means for 
combining the points of the multi-dimensional image selected from the projection values 
fit)m at least one group selected fitjm the first set of project paths with the projection 
values from at least one group selected from a second set of projection paths. 

68. The apparatus according to claim 67 fiuther comprising: 
means for selecting at least one group of projection values; 
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means for selecting the points of the multi-dimensional image corresponding to 

the projection values of the selected group; and 
means for generating an image based upon the selected multi-dimensional image 

points. 

5 69. The apparatus according to claim 56 further comprising means for 

manipulating image groups for enhanced display. 

70. The apparatus according to claim 69 wherein the image manipulation 
means renders the image in hollow image structures. 

7 1 . The apparatus according to claim 56 further comprising a display coupled 
10 to the grouping means to display the processed image. 

72. The apparatus according to claim 69 wherein the image manipulation 
means performs shading of volume surfaces. 

73 . The apparatus according to claim 56 wherein the multidimensional image 
set is a magnetic resonance derived image. 

1 5 74. The apparatus according to claim 56 wherein the multidimensional image 

set is a computed tomography derived image. 

75 . The apparatus according to claim 56 wherein the multidimensional image 
set is a rotational X-ray computed tomography image. 

76. The apparatus according to claim 56 wherein the grouping means 
2 0 performs densiometric summation of the selected number of projection values. 

77. An image processing apparatus comprising: 

means for acquiring a multi-dimensional image of a given subject; 
means, coupled to the acquiring means, for defining a set of projection paths 
through the multi-dimensional image; 

2 5 means for selecting at least one point, along each projection bath, based upon a 

specified criterion; 

means for formulating an array of projection values corresponding to the position 

of selected points along their respective projection path; 
means for grouping a selected number of projection values based upon their 

3 0 proximity to other projection values; and 

means, coupled to the grouping means, for displaying an image based upon image 
points associated with the grouped projection values , 

78. The apparatus according to claim 77 wherein the specified criterion is a 
selected threshold value. 



35 



wo 00/63831 



PCTAJSOO/10532 



25 

79. The apparatus according to claim 78 wherein the threshold value is an 
average background value of all points in a subset of points selected from the multi- 
dimensional image. 

80. The apparatus according to claim 77 wherein the specified criterion is a 
maximum value along the projection path. 

8 1 . The apparatus according to claim 77 wherein the specified criterion is a 
minimum value along the projection path. 

82. The apparatus according to cl^m 77 further comprising: 
means for selecting at least one group of projection values; and 

means for selecting the points of the multi-dimensional image corresponding to 
the projection values of the selected group, wherein the display means 
displays an image based upon the selected multi-dimensional image 
points. 

83 . The apparatus according to claim 77 wherein the multidimensional image 
set is a magnetic resonance derived image. 

84. The apparatus according to claim 77 wherein the multidimensional image 
set is a computed tomography derived image. 

85 . The apparatus according to claim 77 wherein the multidimensional image 
set is a rotational X-ray computed tomography image. 
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